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ABSTRACT 



Techniques for determining an output rate for a bit stream, i 
the output rate being determined by applying information 
read from the bit stream to a model of a receiver for the bit * 
stream. The techniques are employed to construct a statis- 
tical multiplexer that multiplexes varying bit-rate MPEG-2 
bit streams onto a satellite up-link. Minimum and maximum 
output rates for each MPEG-2 bit stream are determined 
such that neither a queue for the bit stream in the multiplexer 
nor the bit stream's decoder will underflow or overflow. The 
rates with regard to the decoder are determined using timing 
information read from the bit stream and a general model of 
an MPEG-2 decoder. The multiplexer first allocates each bit 
stream its minimum bandwidth and then allocates any 
remaining bandwidth to the bit streams in proportion to the 
difference between the minimum and maximum output rates 
for the bit streams, with no bit stream receiving more than 
its maximum output rate. If there is not enough bandwidth 
to give every bit stream its minimum rate, bit rates are 
allocated according to priorities assigned to the bit streams. 

5 Claims, 10 Drawing Sheets 
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USING A RECEIVER MODEL TO temporal redundancy than a sequence of images for a 

MULTIPLEX VARIABLE-RATE BIT commercial or MTV song presentation, and the bit rate for 

STREAMS HAVING TIMING CONSTRAINTS the images showing the "tauing head" will be far lower than 

the bit rate„for the images of the MTV song presentation. 

This application is a divisional of application Ser. No. ^ / Trie MPEG-2 compression scheme represents a sequence^ 

08/823,007 filed Mar. 21, 1997, now U.S. Pat. No. 6,052,384 of video images as a sequence of pictures, each of which 

-7 must be decoded at a specific time. There are three ways in 

BACKGROUND OF THE INVENTION which pictures may be compressed. One way is intra-coding, 



1. Field of the Invention 



in which the compression is done without reference to any 
10 other picture. This encoding technique reduces spatial 
The invention has to do with the transmission of variable- redundancy but not time redundancy, and the pictures result- 
rate bit streams generally and more particularly with the mg f rom £ t are generally larger than those in which the 
efficient time multiplexing of several such bit streams onto encoding reduces both spatial redundancy and temporal 
a transmission medium. redundancy. Pictures encoded in this way are called 
2. Description of the Prior Art 15 I-pictures. A certain number of I-pictures are required in a 
FIGS. 1-3 sequence, first, because the initial picture of a sequence is 
A new problem in data transmission is the transmission of necessarily an I-picture, and second, because I-pictures 
data that requires a high band width, is bursty, and has permit recovery from transmission errors, 
temporal constraints. Traditionally, data transmission has Time redundancy is reduced by encoding pictures as' a set 
been done on the public switched networks provided by the 20 of changes from earlier or later pictures or both. In MPEG-2, 
telephone companies and on packet networks. The public this is done using motion compensated forward and back- 
switched networks are designed for interactive voice ward predictions. When a picture uses only forward motion 
applications, and so provide relatively low-bandwidth cir- compensated prediction, it is called a Predictive-coded 
cuits that satisfy stringent temporal constraints. The packet picture, or P picture . When a picture uses both forward and 
networks are designed for the transfer of data between 25 backward motion compensated predictions, it is called a 
computer systems. The only constraint is that the data Bidirectional predictive -coded picture, or a B picture in 
eventually arrive at its destination. The amount of bandwidth short. P pictures generally have fewer bits than I pictures and 
available for a transfer depends on the degree of congestion B pictures have the smallest number of bits. The number of 
in the network. The packet networks thus typically make no bits required to encode a given sequence of pictures in 
guarantees whatever about when or even in what order the 30 MPEG-2 is thus dependent on the distribution of picture 
data in a burst of data will arrive at its destination. As may coding types mentioned above, as well as the picture content 
be seen from the foregoing, neither the telephone network itself As will be apparent from the foregoing discussion, the 
nor the packet network is well-adapted to handle high- sequence of pictures required to encode the images of the 
bandwidth bursty data with time constraints. An example of "talking heads" will have fewer and smaller I pictures and 
such data is digital television which has been compressed 35 smaller B and P pictures than the sequence required for the 
according to the MPEG-2 standard. For details on the MTV song presentation, and consequently, the MPEG-2 
standard, see Background Information on MPEG-1 and representation of the images of the talking heads will be 
MPEG-2 Television Compression, which could be found in much smaller than the MPEG-2 representation of the images 
November 1996 at the URL of the MTV sequence. 

FIG. 1 shows those details of the MPEG-2 standard that 40 The MPEG-2 pictures are being received by a low-cost 

are required for the present discussion. The standard defines consumer electronics device such as a digital television set 

a encoding scheme for compressing digital representations or a set- top box provided by a CATV service provider. The 

of video. The encoding scheme takes advantage of the fact low cost of the device strictly limits the amount of memory 

that video images generally have large amounts of spatial available to store the MPEG-2 pictures. Moreover, the 

and temporal redundancy. There is spatial redundancy 45 pictures are being used to produce moving images. The 

because a given video picture has areas where the entire area MPEG-2 pictures must consequently arrive in the receiver in 

has the same appearance; the larger the areas and the more the right order and with time intervals between them such 

of them there are, the greater amount of spatial redundancy that the next MPEG-2 picture is available when needed and 

in the image. There is temporal redundancy because there is there is room in the memory for the picture which is 

often not much change between a given video image and the 50 currently being sent. In the art, a memory which has run out 

ones that precede and follow it in a sequence. The less the of data is said to have unde41owed, while a memory which, 

amount of change between two video images, the greater the has received more data than it can hold is said to have 

amount of temporal redundancy. The more spatial redun- overflowed In the case of underflow, the motion in the TV 

dancy there is in an image and the more temporal redun- picture must stop until the next MPEG-2 picture arrives, and 

dancy there is in the sequence of images to which the image 55 in the case of overflow, the data which did not fit into 

belongs, the fewer the bits that will be needed to represent memory is simply lost. 

the image. FIG. 1 is a representation of a digital picture source 103 

^Maximum- advantage for the transmission of images and a television 117 that are connected by a channel 114 that 

encoded using the MPEG-2 standard is obtained if the / is carrying a MPEG-2 bit stream representation of a 

images can be transmitted at variable bit rates. The bit rates' 60 sequence of TV images. In system 101, a digital picture 

can vary because the rate at which a receiving device source 103 generates uncompressed digital representations 

receives images is constant, while the images have varying of images 105, which go to variable bit rate encoder 107. 
number of bits. A large image therefore requires a higher bik Encoder 107 encodes the uncompressed digital representa- 

rate than a small image, and a sequence of MPEG images tions to produce variable rate bit stream 109. Variable rate bit 

transmitted at variable bit-rates is a variable-rate bit stream 65 stream 109 is a sequence of compressed digital pictures 111 

with time constraints. For example, a sequence of images of variable length. As indicated above, when the encoding is 

that shows a "talking head" will have much more spatial and done according to the MPEG-2 standard, the length of a 
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picture depends on the complexity of the image it represents 
and whether it is an I picture, a P picture, or a B picture. 
Additionally, the length of the picture depends on the 
encoding rate of VBR encoder 107. That rate can be varied. 
In general, the more bits used to encode a picture, the better 5 
the picture quality. 

Bit stream 109 is transferred via a channel 114 to VBR 
decoder 115, which decodes the compressed digital pictures 
111 to produce uncompressed digital pictures 105. These in 
turn are provided to television 117. If television 117 is a 10 
digital television, they will be provided directly; otherwise, 
there will be another element which converts uncompressed 
digital pictures 105 into standard analog television signals 
and then provides those signals to television 117. There may 
of course be any number of decoders 115 receiving the 15 
output of a single encoder 107. 

In FIG. 1, channel 114 transfers bit stream 109 as a 
sequence of packets 113. The compressed digital pictures 
111 thus appear in FIG. 1 as varying-length sequences of 
packets 113. Thus, picture lll(rf) has n packets while picture 20 
111(a) has k packets. Included in each picture 111 is timing 
information 112. Timing information 112 contains two kinds 
of information; clock information and time stamps. Clock 
information is used to synchronize decoder 115 with encoder 
107. The time stamps specify when a picture is to be 25 
decoded and when it is actually to be displayed. The times 
specified in the time stamps are specified in terms of the 
clock information. As indicated above, VBR decoder con- 
tains a relatively small amount of memory for storing 
pictures 113 until they are decoded and provided to TV 117. 30 
This memory is shown at 119 in FIG. 1 and is termed in the 
following the decoder's bit buffer. Bit buffer 119 must be at 
least large enough to hold the largest possible MPEG-2 
picture. Further, channel 114 must provide the pictures 111 
to bit buffer 119 in such fashion that decoder 115 can make 35 
them available at the proper times to TV 117 and that bit 
buffer 119 never overflows or overflows. Bit buffer 119 
underflows if not all of the bits in a picture 111 have arrived 
in bit buffer 119 by the time specified in the picture's time 
stamp for decoder 115 to begin decoding the picture 111. 40 

Providing pictures 111 to VBR decoder 115 in the proper 
order and at the proper times is made more complicated by 
the fact that a number of channels 114 may share a single 
very high bandwidth data link. For example, a CATV 
provider may use a satellite link to provide a large number 45 
of TV programs from a central location to a number of 
CATV network head ends, from which they are transmitted 
via coaxial or fiber optic cable to individual subscribers or 
may even use the satellite link to provide the TV programs 
directly to the subscribers. When a number of channels share 50 
a medium such as a satellite link, the medium is said to be 
multiplexed among the channels. 

FIG. 2 shows such a multiplexed medium. A number of 
channels 114(0) through 114(n) which are carrying packets 
containing bits from variable rate bit streams 109(0 . . . n) 55 
are received in multiplexer 203, which processes the packets 
as required to multiplex them onto high bandwidth medium 
207. The packets then go via medium 207 to demultiplexer 
209, which separates the packets into the packet streams for 
the individual channels 114(0 . . . n). Asimple way of sharing 60 
a high bandwidth medium among a number of channels that 
are carrying digital data is to repeatedly give each individual 
channel 114 access to the high bandwidth medium for a short 
period of time, termed herein a slot. 

One way of doing this is shown at 210 in FIG. 2. The short 65 
period of time appears at 210 as a slot 213; during a slot 213, 
a fixed number of packets 113 belonging to a channel 114 



may be output to medium 207. Each channel 114 in turn has 
a slot 213, and all of the slots taken together make up a time 
slice 211. When medium 207 is carrying channels like 
channel 114 that have varying bit rates and time constraints, 
slot 213 for each of the channels 114 must output enough 
packets to provide bits at the rate necessary to send the 
largest pictures 111 to channel 114 within channel 114' s 
time, overflow, and underflow constraints. Of course, most 
of the time, a channel's slot 213 will be outputting fewer 
packets than the maximum to medium 207, and sometimes 
may not be. carrying any packets at all. Since each slot 213 
represents a fixed portion of medium 207's total bandwidth, 
any time a slot 213 is not full, a part of medium 207's 
bandwidth is being wasted. 

In order to avoid wasting the bandwidth of medium 207, 
a technique is used which ensures that time slice 211 is 
generally almost full of packets. This technique is termed 
statistical multiplexing. It takes advantage of the fact that at 
a given moment of time, each of the channels in a set of 
channels will be carrying bits at a different bit rate, and the 
bandwidth of medium 207 need only be large enough at that 
moment of time to transmit what the channels are presently 
carrying, not large enough to transmit what all of the 
channels could carry if they were transmitting at the maxi- 
mum rate. The output of the channels is analyzed statisti- 
cally to determine what the actual maximum rate of output 
for the entire set of channels will be and the bandwidth of 
medium 207 is sized to satisfy that actual peak rate. 
Typically, the bandwidth that is determined in this fashion 
will be far less than is required for multiplexing in the 
manner shown at 210 in FIG. 2. As a result, more channels 
can be sent in a given amount of bandwidth. At the level of 
slots, what statistical multiplexing requires is a mechanism 
which in effect permits a channel 114 to have a slot in time 
slice 211 which varies in length to suit the actual needs of 
channel 114 during that time slice 211. Such a time slice 211 
with varying-length slots 215 is shown at 214. 

One method of statistically multiplexing bit streams is 
disclosed in Rao, U.S. Pat. No. 5,506,844, Method for 
Configuring a Statistical Multiplexer to Dynamically Allo- 
cate r CbmmunicatioD Channel Bandwidth, issued Apr. 9, 
)6. FIG. 3^isln^averview of the method disclosed in the 
Rao appHcation^JJystem 301 includes a set of encoders 
*02(0 . . . u) Which encode a set of bit streams 105(0 . . . n). 
During a given period of time, termed herein a window, each 
encoder 302(f) encodes at a constant bit rate; however, the 
bit rate may be changed at the beginning of the window. The 
output of an encoder 302(0 is thus a bit stream 108(0 having 
a piecewise-constant bit rate. The bit streams 108(0 . . . n) 
are input to multiplexer 303, which multiplexes them onto 
medium 207. 

Multiplexer 303 maximizes the use of medium 207 by 
adjusting the bit rates of encoders 302(0 . . . n). As 
mentioned above, there is a relationship between bit rate and 
picture quality. Generally, the higher the bit rate, the better 
the picture quality. Consequently, in adjusting the bit rates of 
encoders 302(0 . . . n), multiplexer 303 must be aware of the 
current picture quality of each bit stream and must adjust the 
bit rates not only to maximize the use of medium 207, but 
also to maximize the picture quality of each of the bit 
streams 108(f). 

As Mux 303 operates, it receives information from each 
encoder 107(f) that indicates the picture distortion rale for 
encoder 107(i)'s current encoding rate (DIF 311 (0) and also 
keeps track of the fullness of encoding buffer 307(0 in 
encoder 107(0, as shown by arrow EBF 309(t'). Encoding 
buffer 307(i) holds bit stream 105(0 while it is being 
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encoded, and encoder 107(f) must encode at a rate such that remaining bandwith for a bit stream is done proportionally 

encoding buffer 307(t) neither overflows nor underflows. to the difference between the maximum and minimum rates 

Multiplexer 303 determines from the current distortion rates for the bit stream. 

of the encoders 107 which encoders need to encode at a \ Q another aspect of the invention, the multiplexer further 

higher bit rate and which can encode at a lower bit rate and 5 uscs priority information about the bit streams in adjusting 

at the beginning of a window, it adjusts the rate of each me rates of the bit streams when there is not enough 

encoder 107, as indicated by the arrows BRCIL 305(0 . . . bandwidth for all of the bit streams to receive the minimum 

n), to maximize the picture quality for all of the encoders amount. Each bit stream is either a priority one, priority two, 

107 while maximizing the degree to which medium 207*s or priority three bit stream, with priority one bit streams 

bandwidth is used. When multiplexer 303 reduces or 10 being required to meet the time constraints set in the bit 

increases an encoder 107(t)'s bit rate, it also reduces or stream, priority two bit streams being required only not to 

increases the size of EBUF 307(f) in the encoder. underflow, and priority three bit streams being permitted to 

While the statistical multiplexer of Rao does maximize underflow. Where bandwidth is necessary for priority one or 

the degree to which medium 207's bandwidth is used, it has priority two bit streams, it is taken from priority two and/or 

a number of shortcomings. Perhaps the most important of is priority three bit streams 

these is that it adjusts the multiplexing by changing picture sm of me invention fa a muM lexef for 

quahty. The system thus cannot guarantee any user a given MpE(J _ 2 ^ ^ multipkxcr multiplexcs 

quality of picture. aplurality of MPEG-2 bit streams and constant bit rate audio 

Another shortcoming is that it requires encoders that bu strMms omo „ medium , ncluded m ^ 

encode digital images as piecewise-constant bit streams. 20 of ^ mventkm are , receiver model for mpeG-2 bit 

Such bit streams have a lower degree of compression than s(reanjs and ^ ritbms for usi , he mode , t0 det6rmin6 the 

variable-rate bit streams; further, toe encoding rate and ou( bft ut) for a MpEG _ 2 bu stream 

therefore the quality of the picture changes at the beginning JL JL , 

of each window; with sequences of fast changing images, , ^ and other ^ cts objects of the mvenUon will 

this will produce coding artifacts in the pictures. 25 bcconie a PP arent t0 those skai f d ^ to w * ich 

Still another is that the multiplexing requires feedback Jfenuon P«*u» upon perusal of the following Detailed 

from multiplexer 303 to encoders 302(0 . . . n). One Description and Drawing, wherein: 

consequence of this fact is that multiplexer 303 will not p nF c rRTPTrn vr OF nRAWTMfiS 

work with pre-stored sequences of pictures 111; another is BRIEF DESCRIP ™ N OF THE DRAWINGS 

that in order to use information like encoder buffer fullness 30 FIG. 1 is a diagram showing how digital television 

309 and distortion information 311 to allocate bandwidth in pictures are encoded, transmitted, and decoded; 

medium 207, multiplexer 303 must take into account the FIG. 2 is a diagram showing multiplexing of variable-rate 

inner workings of encoder 107. A third is that there must be bit streams onto a ^gh band width medium; 

a high-speed connection between multiplexer 303 and each rj „ ~ . «. , - i 

encoder 302 to exchange the control information. Finally, 35 3 13 a block dia « ram of 4 P 00 ™ 1 staUsUcal multI - 

the bitrate switching of the encoders and the multiplexer is p exer ' 

difficult to implement, particularly if it is necessary to FIG - 4 ™ a block diagram of a statistical multiplexer 

support video inputs having different frame rates. which implements a preferred embodiment of the invention; 

It is an object of the invention disclosed herein to over- FIG. 5 is a more detailed block diagram of a part of the 

come these shortcomings and thereby to provide an 40 statistical multiplexer of FIG. 4; 

improved statistical multiplexer. FIG. 6 is pseudo-code for the algorithm used to determine 

SUMMARY OF THE INVENTION ^ e ^ ra * e °^ a channel in the preferred embodiment; 

. . . , ., , . , . FIG. 7 is a flow chart for the algorithm used to allocate the 

Hie invention is embodied in a multiplexer that over- ^ bU ^ of medium m m me 

comes the foregoing problems of the prior art by determin- 45 „ . , < . , ,r „ . ... 

ing the rate at which each bit stream is to be output to the F1G - 8 18 a conceptual block diagram of the statistical 

medium in accordance with rate information obtained by mu m cxcx * 

applying a model of a receiver for each of the bit streams to FIG- 9 is. a high-level block diagram of an encoding 

information contained in the bit stream itself. The rate for system which includes an implementation of the statistical 

each bit stream is determined such that the receiver for the 50 multiplexer; 

bit stream will neither underflow nor overflow and the FIG. 10 is a more detailed view of the implementation of 

multiplexer's use of the medium's bandwidth is maximized. the statistical multiplexer; 

Advantages of the multiplexer of the invention include the FIG. 11 is a detailed view of a channel input block in the 

fact that all of the information needed to determine the rates statistical multiplexer of FIG. 10; and 

is available within the multiplexer itself, the fact that the 55 Fia n is a flowchart of the minimal bitratc algorithm. 

technique will work with any kind of bit stream for which it w 

is possible to make a receiver model that is able to determine J . 71,(5 r f[ erence "™? CB 10 draWlngS h / VC at leaSt 

an output rate on the basis of information read from the bit ™ e two nghtmost digits are reference numbers 

stream, and the fact that adjustment of the output rales does wlthm a ' he ^ l ° lhe ^ of lho f dl f* « re lhe 

not affect the contents of the bit stream. 60 T of ^ L fi ^ re ,n which * he ,tem ldentlfled b y '. he 

7 t #1 _ ♦ • r *• <* ^ «u uu ~ ™ reference number first appears. For example, an item with 

In another aspect, the rate information for each bit stream - . . „ _f ' 

, r ; . n , j „ ■ reference number 203 first appears in FIG. 2. 

is provided as a minimum allowable rate and a maximum rr 

allowable rate and the multiplexer allocates bandwidth to the DETAILED DESCRIPTION 
bit streams by first giving each bit stream its minimum 

allowable rate and then allocating any remaining bandwidth 65 The following Detailed Description will first present an 

to the bit streams up to the maximum rate for each bit overview of the preferred embodiment, will then provide a 

stream. In the preferred embodiment, allocation of the description of the hardware in which the preferred embodi- 
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merit is implemented, and will finally provide a detailed controller 407(0- It should be noted here that the packets in 

description of the algorithms used to allocate bandwidth in each bit stream 109(i) arrive in transmission controller 

the preferred embodiment. 407(0 in the order in which they were sent by encoder 

Conceptual Overview: FIG. 8 107(i> 

FIG. 8 presents a conceptual overview of a statistical 5 Transmission controller 407(0 determines the rate at 

multiplexer 801 which incorporates the principles of the w ™*ch packets from its corresponding bit stream 109(0 is 

invention. A number n of variable-rate bit streams 109 are output to medium 207, The actual rate determination is made 

received in receiver 803, which provides them to bandwidth by transmission rate controller 413, which at a minimum, 

portion controller 805. Bandwidth portion controller 805 bases its determination on the following information: 

dynamically determines what portion of the bandwidth of 10 for at least a current picture 111 in bit stream 109(0, me 

medium 801 that each bit stream 109(7) is to receive and timing information 112 and the size of the current 

provides a corresponding portion 815(0 of the bit stream to picture. 

transmitter 817, which outputs the portions 815(0 ... n) it a Video Buffer Verifier (VBV) model 415(0, which is a 

receives of each bit stream 109(0 . . . n) onto medium 207. model of a hypothetical bit buffer 119(i). 

Bandwidth portion controller 805 has a number of sub- 15 VBV model 415(t) uses the timing information and picture 

components. There is a transmission controller 807(0 for size information to determine a range of rates at which bit 

each bit stream 109(f). Each transmission controller 807(0 stream 109(0 must be provided to the decoder's bit buffer 

contains a bit stream analyzer 809(0 and a receiver model 119(f) if bit buffer 119(i) is to neither overflow nor under- 

811(i). Eit stream analyzer 809(0 collects information from flow. Transmission rate controller 413(0 provides the rate 

bit stream 109(0 and applies receiver model 811(0 to me 20 information to packet delivery controller 419, which uses 

collected information to determine what rate is required by the information from all of the transmission controllers 407 

the condition of the receiving device. In the case of a to determine during each time slice how the bandwidth of 

MPEG-2 bit stream, the receiving device is a decoder 115(0; transmission medium 207 should be allocated among the bit 

and for such a decoder, the required rate can be determined streams 109 during the next time slice. The more packets a 

from the time stamps and the sizes of the pictures making up is bit stream 109(0 needs to output during a time slice, the 

bit stream 109(0- Transmission controller 807(0 applies more bandwidth it receives for that time slice, 

receiver model 811(0 t0 ^ information to determine rate Continuing in more detail transmission controller 407 

information 812(0- Bandwidth allocator 813 receives rate obtains the timing and picture size information by means of 

information 812(0 . . . n) and uses this information to bit stream analyzer 409, which reads bit stream 109(0 ^ ^ 

allocate the portion of the bandwidth of medium 207 that 30 enters transmission controller 407 and recovers the timing 

each bit stream 109(0 is to receive. Having done this for information 114 and the picture size 411 from bit stream 

each bit stream 109(0 . . . n), it provides a bit stream portion 109(0- Bit stream analyzer 409 can do so because the 

815(0 that corresponds to the allocated bandwidth to trans- MPEG-2 standard requires that the beginning of each picture 

mitter 817. Ill be marked and that the timing information 114 occupy 

It is worth noting here that all of the information required 35 predetermined locations in each picture 111. As previously 

by the above technique for allocating bandwidth can be explained,, timing^ information 114 for each picture 111 

obtained by applying the receiver models 811 to the infor- includes a clock value and a decoding time stamps Trans- \T/ 

mation received from the bit streams 109 and that informa- mission" controller 407(0 arid later decoder 115(0 use the 

tion need only be exchanged between bandwidth allocator clock value to synchronize themselves with encoder 107(0- 

813 and transmission controllers 807. There is no need 40 The timing information is found in the header of the PES 

whatever to receive information from or provide information packet that encapsulates the compressed video data. The 

to the encoders 107. Put another way, all of the information information is contained in the PTS and DTS time stamp 

needed to allocate the bandwidth is available within statis- parameters of the PES header. The MPEG-2 standard 

tical multiplexer 801 itself. requires mat a time stamp be sent at least every 700 msec. 

It is also worth noting that the technique of using a model 45 If a compressed picture is not explicitly sent with a com- 

of a receiver to control the rate at which a bit stream is output pressed picture, then the decoding time can be determined 

to a receiver may be applied in other situations. For example, from parameters in the Sequence and Picture headers. For 

a receiver model could be used to control the rate at which details, see Annex C of ISOAEC 13818-1. Bit stream 

a MPEG-2 encoder encoded data. anajg zcr » 409 - do tomines the size of a picture simply by 

Overview of a Preferred Embodiment: FIG. 4 5(/^owiting the bits^r packets) from the beginning of one 

FIG. 4 provides an overview of a statistical multiplexer vpictu^iaihe-begmning of the next picture. 

401 for MPEG-2 bit streams which is implemented accord- The timing information and the picture size are used in 

ing to the principles of the invention. The main components VBV model 415(0- V **V model 415(0 requires the timing 

of multiplexer 401 are packet collection controller 403, a information and picture size information for each picture in 

transmission controller 407(0 for each variable-rate bit 55 bit su-eam 109(0 from the time the picture enters multiplexer 

stream 109(0, a packet delivery controller 419, and a 401 until the time the picture is decoded in decoder 115(i).Jlhj 

modulator 423, which receives the output of packet delivery DTS buffer 414 must be large enough to hold the timing\l/ 

controller 419 and outputs it in the proper form for trans- information for all of the pictures required for the model. It 

mission medium 207, Packet collection controller 403 col- should be noted here that VBV model 415(0 s behavior is 

lects packets from variable -rate bit streams 109(0 . . . n) and 60 defined solely by the semantics of the MPEG-2 standard, not 

distributes the packets that carry a given bit stream 109(0 to by any concrete bit buffer 119(0- Any bit buffer for a 

the bitream's corresponding transmission controller 407(0- working MPEG-2 decoder must be able to provide the 

In the preferred embodiment, the packets for all of the bit decoder with the complete next picture at the time indicated 

streams 109(0 . . . n) are output to bus 402. Each packet by the picture's timing information; that means that the bit 

contains an indication of which bit stream it belongs to, and 65 buffer 119(0 for any working MPEG-2 decoder must be at 

packet collection controller responds to the indication con- a minimum large enough for the largest possible MPEG-2 

tained in a packet by routing it to the proper transmission picture. Given this minimum buffer size, the timing infor- 
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mation r forjhe_pictures,-ajid^the sizes of the individual 
pictures, VBV model 415(f) can determine a rate of output 
\ for Sit stream 109(i) which will guarantee for bit buffers 
^19(j) of any working MPEG-2 decoder that each picture 
- "arrives in the bit buffer 19(i) before the time it is to be 
decoded and that there will be jio overflow of bitjmffery 
119(0. "" 

Details of Transmission Controller 407 and Packet Delivery 
Controller 419: FIG. 5 

FIG. 5 shows the details of a preferred embodiment of 
transmission controller 407 and packet delivery controller 
419. The figure shows three of the n transmission 
controllers, namely transmission controllers 407(i . . . k), and 
the two major components of packet delivery controller 419, 
namely central bit rate controller 501 and switch 511. 
Beginning with transmission controller 407(0, in addition to 
transmission rate controller 413, analyzer 409, and VBV 
model 415, transmission controller 409 includes statistical 
multiplexer buffer (SMB) 507, a meter 505 for buffer 507, 
and throttle 509. 

SMB 507(f) is a first-in-first-out pipe buffer which holds 
the bits of bit stream 109(i) while they are in transmission 
control 407(i). In* the preferred embodiment, SMB 507(0 
receives pictures 111 in bursts that contain all or almost all 
of the bits in the picture, depends on the picture size and 
maximal bit rate specified by the encoder. Such bursts are 
termed herein picture pulses, and the time period represented 
by such a picture pulse is denoted as T^, which is the inverse 
of video frame rate. For example, T p = 1/29.97=33 ms for 
NTSC video coding. As previously stated, packet delivery 
controller 419 provides packets in time slices 211. The 
length of time of one of these slices is denoted herein as T c . 
In a preferred embodiment, T c , is 10 ms. 

SMB 507(0 naust of course be large enough to be able to 
accept picture pulses of any size during the time it takes to 
read out the largest expected picture pulse. SMB 507(0 
further must be emptied at a rate that ensures that it cannot 
overflow, since that would result in the loss of bits from bit 
stream 109(0- I* should not underflow, since that would 
result in the insertion of null packets in the bit stream, 
^rrmltin^ in the waste of a portion of the multiplexed 
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medium. Meter 505 monitors the fullness of SMB 507(0 and 
provides information concerning the degree of fullness to 
JE&C413(0. TRC 413(0 then uses this information to vary 
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the range of bit rates that it provides to packet delivery 
controller 419 as required to keep SMB 507(0 from over- 
flnwjnff nr upH^ rjWi/ing In other embodiments, the degree, 
pf-fullness from meter 505 can also be fed back to encoder 
107(0 and used there to increase or decrease the encoding 
rate. It should be noted here that feeding back the degree of 
-fullness to encoder 107(0 does not create any dependencies 
between statistical multiplexer 401 and a given type of 
encoder 107. Throttle 509, finally, is set by TRC 413 on the 
basis of information 418(0 tnat i* nas received from packet 
delivery controller 419 to indicate the number of packets 113 
that bit stream 109(0 is t0 provide to medium 207 in time 
slice 211. 

In determining the range, TRC 413 sets the minimum rate 
for a given time slice 211 to the maximum of the rate 
required to keep SMB 507 from overflowing and the rate 
required to keep VBV model 415(0 fr° m underflowing and 
the maximum rate for the time slice to the minimum of the 
rate required to keep SMB 507 from underflowing and the 
rate required to keep VBV model 415(0 from overflowing. 

Continuing with packet delivery controller 419, packet 
delivery controller 419 allocates the packets 113 that can be 
output during the time slice 211 T e to bit streams 109(0 . . . n) 
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f as required to simultaneously satisfy the ranges of rates and 
, ' priorities provided by TRC 413 for each transmission con- 
troller 407(0 a °d maximize the number of packets 113 
output during time slice 211. In the preferred embodiment, 
controller 419 has two components, central bit rate control- 
ler 501, which is a processor that analyzes the information 
received from each of the transmission rate controllers 413 
in order to determine how many packets from each bit 
stream 109(0 are to be output in the next time slice 211, and 
switch 511, which takes the number of packets 113 permitted 
by throttle 509(0 for eacn bit stream 109(0 during the time 
slice 211. Switch 511 is implemented so as to deliver packets 
from each throttle 509(0 sucn tnat the packets ^re evenly 
distributed across time slice 211. Implementing switch 511 
in this way reduces the burstiness of the stream of packets 
109(0 10 dec °der 115(0 and thereby reduces the amount of 
transport packet buffer needed in decoder 115. Such imple- 
ment ationsljf switch 511 are well-known in the art. 

An important advantage of multiplexer 401, or indeed of 
any statistical multiplexer built according to the principles of 
the invention is that the multiplexer can simultaneously 
multiplex both constant- rate and variable-rate bit streams 
onto medium 207. The reason for this is that as far as 
statistical multiplexer 401 is concerned, a constant-rate bit 
stream is simply a degenerate case: it is a varying-rate bit 
stream whose rate never varies. Thus, with a constant-rate 
bit stream, TRC 413(0 always returns the same rate infor- 
mation 417(0 t0 packet delivery controller 419. 
Hardware Implementation of a Preferred Embodiment: 
FIGS. 9-11 

A presently-preferred embodiment of the invention is 
implemented as a modification of the PowerVu satellite 
up-link system manufactured by Scientific-Atlanta, Inc. 
(PowerVu is a trademark of Scientific-Atlanta). FIG. 9 is a 
high-level block diagram of the Power- Vu up-link system as 
modified to implement the invention. System 901 includes a 
set of encoders 911(0 . . . n). Each encoder 911(0 encodes 
a video input 903(0 ^d an au dio input 905(0; tne video 
input is encoded at ^constant or variable bit rate and the 
audio input is encoded at a constant bit rate. Each encoder 
911(0 nas an output 913(0 which carries the encoded video 
and audio. In the PowerVu system as modified, the outputs 
913(0 ... n) go to statistical multiplexer 915, which outputs 
a constant bit-rate stream 917 to a modulator for transmis- 
sion to a communications satellite. At a high level, operation 
of all of the components of system 901 is supervised and 
controlled by control processor 907, which communicates 
with the other components by means of^Ethernet protocol 
909 (Ethernet is a registered trademark of Xerox 
Corporation). In the presently-preferred embodiment, statis- 
tical multiplexer 915 is implemented as a separate chassis 
which need only be coupled to the rest of the PowerVu 
system by encoded data inputs 913(0 . . . n), Ethernet 
protocol 909, and output 917. \ 

FIG. 10 shows the preferred embodiment of statistical 
multiplexer 915 in more detail. Multiplexer 915 receives its 
inputs of encoded video and audio from optical fibers. Each 
SWIF receiver 1001(0 receives input from a single optical 
fiber and there are receivers 1001(0 . . . n) corresponding to 
encoders 911(0 . . . n). Each receiver converts the informa- 
tion from photons to digital electronic form and outputs it 
via PCR MOD 1005(0 to channel input block 1009(0- PCR 
MOD 1005(0 corrects the clock information in the encoded 
video and audio to compensate for any delays in the encod- 
ing process. The synchronization information needed to do 
this is provided by MSYNC lock up 1003. 

Channel Input 1009(0 is an implementation of transmis- 
sion controller 407(0- Channel input 1009(0 employs a 
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software implementation of VBV model 415 to dynamically As shown in FIG. 11, packet director 1101 is implemented 

determine a current rate at which the input from receiver by means of gate arrays and a dual port RAM memory. 

1001(0 must be output to multiplexed output stream 917 and Counters 1107. and 1123 are also implemented using gate 

provides that rate information to central bit rate controller arra y S and channel controller 1113 is a digital signal pro- 

1007, which in turn actually allocates a specific rate to s cessor. Central bitrate controller 1007 is implemented using 

channel input block 1009(0- Channel input block 1009(0 a microprocessor with a support IC. 

then outputs bits in its bit stream to bus 1011 at that rate. The Detailed Description of Algorithms used to Compute the 

combined outputs of blocks 1^9(0 ^ . n) then go via 0u ^ m Ratc f a Bit strcam 109(0 from Statistical Muld . 

multiplexed output 1013, PCR MOD 1016, and SWIF . 7. 4ft1 . P!r o * 7 flnH 17 

transmitter 1017 to output 917. PCR MOD 1016 modifies 1fl P l ™ ; ^LZ 1 th? m»™™ m ™t, p w h,vh » 
. F . • « • 1 , , , 10 As indicated above, the maximum rate R-™. at which a 
the clock information in the encoded video again to deal 4 . . , .. ams^ * . 1 * 11*1 . 
with the time spent in channel input block 1009® and transmission controller 407(0 may ou put Potato 113 .to 
outputs the bit stream to SWIF transmitter 1017, which ™ d r? 207 ^ determmed by the need to keep , S MB buffer 
converts the bit stream to a photonic representation and f07(0 from underflowing and bit buffer 119(0 from over- 
outputs it to an optical fiber. Communication processor 1015 n ™ n *J^™ n ]F^%^ 15 d ^ termmed b X ^ ™J d 
provides high level control to central bitrate controller 1007 « to keep SMB buffer 507(0 from overflowing and bit buffer 
and also serves as the interface to PCC 907, a control 119 (0 from underflowing. Bit buffer 119(0 ™ n not ™der- 
console, and a system which broadcasts status information. Row if all packets belonging to the picture currently being 
Communications processor 1015 also receives MPEG-2 sent arrive in bit buffer 119(0 before the time indicated in the 
service information tables from PCC 907 and provides them DTS stamp for the picture. 

to service information table insertion 1018, which inserts 20 There are thus two maximum rates and two minimum 

them into the bit streams. rates that need to be taken into account in determining R m(W 

A presently -preferred embodiment of a single channel and R min : 

input block 1009(0 is shown in more detail in FIG. 11. The R ma *i is the maximum rate at which bit buffer 119(0 in any 

main components are packet director 1101, which detects MPEG-2 decoder that conforms to the standard will not 

audio packets, video packets, and headers and routes them to 25 overflow; 

different components of input block 1009(0, storage 1115 R ma *2 is the maximum rate at which SMB 507(0 wiU not 

for the headers, storage 1117 for a FIFO (queue) to hold underflow; 

video packets from the time they are received in input block ^ m ini is me minimum rate at which bit buffer 119(0 not 

1009(0 until they are output to data bus 1011, and a bypass underflow; and 

FIFO 1119 which holds the constant bit rate audio packets 30 R m£ft2 is the minimum rate at which SMB 507(0 will not 

while they are in input block 1009(0- Output from FIFO overflow. 
1117 is controlled by throttle 1032 under control of throttle 

counter 1123, which specifies the number of packets to be and minima as follows: 

output from FIFO 1117 during a given time slot. Output R^ is the minimum of R w 

from FIFO 1127 is controlled by throttle 1129, which is 35 R min is the maximum of R mi -„i and R^-^. 

controlled by throttle counter 1123. Throttle counter 1123 is What is needed to compute R minl and R maxl is a VBV model 

set by channel controller 1113 in response to the rate selected 415(0 that models the fullness and emptiness of bit buffer 

by central bit rate controller 1007. Throttle counter 1127, 119(0; what is needed to compute R min2 and R™^ is a 

which is for a constant -rate bit stream and does not depend measure of the fullness and emptiness of SMB buffer 507(0- 

on VBV model 415(0, is set directly by central bit rate 40 The model for the fullness of bit buffer 119(0 is termed 

controller 1007. herein VBV fullness and the model for the emptiness of bit 

Operation of input block 1009(0 i s as would be expected. buffer 119(0 is termed herein VBV emptiness. The algo- 

Serial bit stream 1001(0 from SWIF receiver 1001(0 is rithms for measuring VBV emptiness and SMB buffer 

modified by PCR 1005(0 and is output to packet director emptiness and fiullness are simple and will be dealt with 

1101, which detects packets, determines their types, and 45 first; the algorithm for measuring VBV fullness is substan- 

outputs them to the various components of channel input tially more complex. 

block 1009(0- Packet director 1101 further provides a start ^ n the case of SMB 507(0, the measure of SMB 

of picture interrupt 1103 to channel controller 1113 to emptiness, E^^, is the amount of free space remaining in 

indicate that a new picture is being received in SMB FIFO SMB 507(0- F° r a given time slice T c 2U(fn), it is defined 

1117. Channel controller 1113 responds to interrupt 1103 by 50 as follows: 
using picture size information obtained from picture counter 

1107, header information stored in header storage 1115, and iWSMB.siZE-z^m) 

information about the amount of space left in SMB FIFO Wner e F SAfB is the actual SMB fullness measured by the 

1117 in the VBV model 415(0 to obt ain maximum and Meter 505. Since there is a maximum size for MPEG-2 

minimum rates at which data must be output from SMB 55 pictures, termed herein VBV_SIZE, the way to prevent 

FIFO 1117 to avoid overflow or underflow in SMB FIFO SMB 507(0 from overflowing is to guarantee that there is 

1117 and overflow or underflow in VBV model 415(0- always an empty space in SMB 507(0 that is larger than or 

Channel controller 1113 outputs these rates via 1121 to equal to VBV_SIZE. If the free space becomes less than 

central bitrate controller 1007, which selects a rate for the th at> th c minimum rate with regard to SMB 507(0, R miH2 > 

next time slice on the basis of the information from channel 60 mus t be increased in the next time slice T c (m+1) according 

controller 1113, the current output requirements of all of the ( 0 mc algorithm below: 

other channel controllers 113, and the total capacity of thc jf ^ <VBV SIZE){ 

output stream. Central bitrate controller 1007 returns the R^%i+1)=^(VBV SIZE-E Jjn))/T ' 



} 



selected rate to channel controller 1113, which sets throttle 
counter 1132 accordingly, throttle counter 1132 then deter- 65 
mines how many bits are actually output by throttle 1125 
during the next time slice. R mMx2 (m+i)~F SMB (m)rr c 
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Continuing with the determination of R m£nJ for the next T c 
from VBV model 415(0, the rate can be found from the 
information in VBV model 415(0 concerning the pictures 
111 in SMB 507(0 . The rule is simply this: the minimum bit 
rate must be such that the picture currently being output is 
completely output from SMB 507(0 before the time indi- 
cated by its DTS time stamp. One implementation is 

Here, pic_residuaL_bits is the number of bits of the picture 10 
111 remaining in SMB 507(0, q is the index of the picture 
currently being transmitted from SMB 507(0 and 
... are the indexes of the following pictures, DTS_V max is 
the time stamp with the most recent time in VBV model 
415(0, and 1 is the actual time determined by the synchro- 15 
nization time value in the bit strean. 

The above algorithm guarantees that all bits belonging to 
the picture 111 which is currently being delivered to bit 
buffer 119(0 wi^ navc been delivered before the decoding 
time DTS_V majc arrives. This algorithm may leave only one 20 
coded picture in the decoder's bit buffer for decoding. While 
this picture could be decoded correctly, a high bit rate will 
be necessary to deliver the next picture on time such that all 
the bits belong to the next picture, p+1, will be available for 
decoding at the next decoding time instance. This require- is 
ment will result in a high bitrate requirement for next Tc 
period and will introduce congestion in the delivery media 
at the next Tc period. A better algorithm is one that guar- 
antees at least two pictures (or more, as long as VBV model 
415(0 does not indicate an overflow) in bit buffer 119(0, 30 
such as the following: 

^«i«i(M+])«pic_residual_bits(g)/(Dr5(9-l)-f) 

In this scheme, the minimal bitrate calculation is slightly 
changed by using the second largest value of DTS in bit 35 
buffer (119(0, DTS (q-1). That is the time stamp for the 
picture 111 preceding the last picture 111 to be sent to bit 
buffer 119(0- This scheme guarantees that the picture p has 
already be delivered to decoder 115(0 at t-DTS (q-1). Of 
course, it is even better to set up the minimal bit rate so that 40 
the number of coded pictures in bit buffer 119(0 is usually 
more than 2. 

Determining VBV Fullness: FIG. 6 

When there is no need to prevent overflow of SMB 507(0, 
the maximum bitrate of bit stream 109(0 is determined from 45 
^he VBV fullness indicated by VBV model 415(0- 
\ The greater the VBV fullness indicated by the model, the 
less the maximum bitrate. At the beginning of the operation 
of modeU415(0, SMB 507(0 is empty and VBV fullness 
indicates that model 415(0. is empty. As soon as bits appear 50 
in SMB 507(0, central bitrate controller 501 begins output- , 
ting them at a predetermined initial rate, for instance, the 
raverage rate for such variable-rate bit streams. As bits are i 
received in SMB 507(0 and output to medium 207, the 
picture information in VBV model 415(0 is updated each 55 
time slice. The newly updated information is used to com- 
pute VBV fullness for the next time slice and the VBV - 
fullness is used in turn to determine the maximum bit rate 
^max\ al which bits will be output on bit stream 109(0 f° r 
the next time period. The computation is the following: 60 

^«i('«+l)-(VBV_SIZE_^ vfrw (myr c 

where F vbv is the VBV fuitness measure provided by VBV 
model 415(0 ^d m an d arc tne current and next time 
slices T c 211. 65 

In the preferred embodiment, the computation of F vfrv (m) 
is governed by the following considerations: 



The calculation requires a computation of the number of 
pictures 111 are currently contained in VBV model 
415(0- 

The calculation requires a knowledge of how many bits of 
the picture 111 which is currently being transmitted 
from SMB 507(0 presently remain in SMB 507(0- 

The data items used to compute F^ (m) in thc preferred 
embodiment include the following: 

a. VBV_SIZE, that is, the maximum size of a MPEG-2 
picture. 

b. The absolute maximum bit rate R,^ which packet 
delivery controller 419 can provide to bit stream 109(0- 

c. The current time, t, recovered from the clock time 
information of bit stream 109(0- 

d. Data items for each picture presently in SMB 507(0: 
packet_cnt, the number of packets 113 in the picture, 
DTS, the time stamp for the picture, q, the index for 
DTS and packet__cnt for the picture currently leaving 
SMB 507(0, ^d r, the index for those values for the 
oldest picture for which there is still information in 
model 415(0- 

e. Status data items in VBV model 415(0 mat arc updated 
every T c 211: pic_cnt__VBV, the number of pictures 
HI which are presently represented in VBV model 
415(0; pic_residual_bit(q), the number of bits of 
picture 111 q that is currently being transmitted to 
decoder 115(0 that remain in SMB 507(0; DTS^V^, 
the time stamp with the most recent time stamp value 
that is presently in VBV model 415(0; ^d F^ itself. 

As soon as SMB 507(0 begins receiving bit stream 109(0, 
packet delivery controller 419 sets throttle 509(0 t0 mc 
initial rate provided by central bit rate controller 509. As 
packets are read from SMB 507(0 at mat ratc > transmission 
rate controller 413(0 updates DTS_V max pic_cnt_VBV, 
F^, and pic_residual_bits (q) as required by the transmis- 
sion of pictures from SMB 507(0 to decoder 115(0 an d by 
the addition of bits to SMB 507(0- The algorithm 601 used 
to do this in a preferred embodiment is shown in FIG. 6. 
Section 603 of algorithm 601 shows how the parameters are 
initialized at the time the first picture arrives in SMB 507(0- 
Execution of loop 604 begins when the first bits of the 
picture arrive in SMB 507(0- As shown at 605, the loop is 
executed once every T c 211. At the beginning of each 
execution of loop 604, pic_residual_bits is decremented by 
the number of bits that were sent at the rate R (m) previously 
determined for the current T tf 211 by central bitrate control- 
ler 501. 

At 607, F vfrv is computed. There are two cases. In the first 
case, shown at 609, the time stamp DTS for the current 
picture r in VBV model 415(0 indicates a time that is after 
the current time t for bit stream 109(0, so decoding of the 
picture r cannot yet have begun. Consequently, the bits that 
were sent during the last T e 211 are simply added to the bits 
that are already in VBV model 415(0 an d FVtv is incre- 
mented by that amount. If the comparison of t and DTS (r) 
indicates that decoder 115(0 nas already begun decoding the 
picture r, the second case, shown at 611, is executed, 
piemen t_VBV is decremented to indicate that one less 
picture is now represented in VBV model 415(0 an d F vfcv is 
adjusted by the difference between the number of bits sent 
to decoder 115(0 i° lnc T 0 211 and the total number of 
bits in the picture that is no longer represented in VBV 
model 415(0- Aft er picture r is removed from VBV model, 
415(0, tne index r is incremented by 1. 

Block of code 613 deals with the updating that has to be 
done when a picture q has been completely read from SMB 
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507(f). When that is the case, pic_residual__bits. will have 
a value that is less than or equal to 0. The first updating that 
has to be done is shown at 615. The time stamp DTS for the 
picture 111 that was just sent is now the maximum DTS in 
Bit buffer 119(0, so DTS_V ffWUr is updated with DTS (q). A 
picture q has also been added to the pictures represented in 
VBV model 415(0, so pic_cnt_VBV is incremented 
accordingly. The second updating is at 617. The new current 
picture is the next picture in SMB 507(0, so q is updated 
accordingly. Similarly, pic_residual_bits is set to the num- 
ber of bits in the new current picture. 
Allocating the Total Capacity of Medium 207 among the 
Channels: FIGS. 7 and 12 

FIG. 7 shows a flowchart 701 of the CBC control algo- 
rithm that is used to assign the new bitrate for each VBR 
encoder for the next Tc period. The control algorithm is a 
loop 713 that executes each T c . At the start of the loop, Rmin 
and Rmax from each TRC(i) are collected. The total avail- 
able bits per Tc parameter, Be, has already been calculated. 
Be will be only updated when there is a change of channel 
bandwidth, Rc, which only happens rarely. Be is calculated 
as 

Bc«Rc*Tc 

where Tc is in units of seconds. 

Be is divided among the bit stream 109 in accordance with 
the ranges of rates specified by the TRCs (0 . . . n) and in 
accordance with a set of priorities which indicate which bit 
streams 109 are more important. The priorities are provided 
by the operator of processor 907 and are set for each bit 
stream when the multiplexer is initialied for the bit stream. 
In the preferred embodiment, there are three levels of 
priority, according to the extent to which timely delivery of 
the pictures in the bit stream is required: 

PL=1: Every picture in the bit stream will be delivered, 
and each of them will be delivered on time. 

PL-2: Some picture will always be delivered on time. For 
example, a picture may be repeated to keep bit buffer 
115(0 fr° m underflowing. 

PL=3: No time guarantees. The bit stream could even be 
interrupted to give the channel to another bit stream, 

PL 1 and 2 are used for real-time video programs. PL 3 is 
used for preemptible data, that is, data which has no real- 
time requirements. Examples of such data are non-real time 
video programs or non-time-dependent data such as E-mail. 
PL 3 permits fuill use of the available bandwidth in situa- 
tions where the sum of the video data is less than the total 
vailable bandwidth. The total bandwidth available that T c : 
and the priority for each bit stream 109(0 k provided by 
input block 707. The total bandwidth, the priorities, and the 
maximums and minimums for the channels are employed in 
block 705 to allocate a minimal bit rate to each bit stream 
109(0. Details on the algorithm used to do this will be given 
below, 

Once the miinlbit rates for all bit stream 109(0 . . . n) have 
been allocated, the algorithm subtracts the allocated bit rates 
from the total bandwidth to determine whether any band- 
width remains (709). If none is left, the allocation is finished 
and as shown at 711. 721, and 715, the bandwidth allocated 
to each TRC 413(0 is assigned to it (721) and loop 713 is 
repeated for the next T c . If there are bits left (branch 717), 
the residual bits are assigned to the bit streams 109(0 that 
can take more bits (719). The algorithm for doing this is also 
explained in more detail below. Once the residual bits have 
been assigned, blocks 701, 715, and loop 213 are executed 
as described above. There remains, of course, the possibility 
that there is not enough total bandwidth to perform the 
allocation of block 705. This worst-case scenario is called 
Panic mode and will be further discussed later. 
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Minimal bitrate allocation algorithm, FIG. 12 

FIG. 12 shows a flowchart 1201 for this algorithm. The 
algorithm allocates a minimal bitrate to each TRC 413(0 aD d 
returns the number of bits still available to be allocated. The 
allocation is ordered by priorities, beginning with PL-1, as 
shown in block 1201. Hie remainder of the flowchart 
consists of an inner loop 1215, which is executed for each 
TRC 413(0 belonging to a given priority and an outer loop 
1233 which is executed for each priority. The algorithm 
terminates when any of three conditions occurs: 
there is no more bandwidth to allocate; 
rates have been allocated to all bit streams 109(0 . . . n); 
allocations have been made for all of the priorities. 
Continuing in more detail with inner loop 1215, in block 
1203, the TRC 413(0 t0 which bandwidth is currently being 
35 allocated receives the amount determined by R^ (i) for that 
TRC 413(0- Tb e bandwidth is rounded to complete 188-bit 
packets. In decision block 1205, it is determined whether 
there is any bandwidth left. If not, branch 207 is taken, 
terminating loop 1215; if there is, loop 1215 continues to 
20 decision block 1211, where it is determined whether there 
are more bit streams 109(0 having the current priority. If 
there are, loop 1215 is repeated; otherwise, as indicated by 
branch 1213, the program enters a new iteration of outer 
loop 1213. In that loop, decision block 1215 first checks 
25 whether there is another priority level to be processed; if 
there is (branch 227), PL is incremented and a new set of 
iterations of inner loop 1215 for that priority begins. If there 
is no additional priority level, loop 1233 terminates, as seen 
at branch 1229. 

Looking at the termination conditions in more detail, if 
there is no more bandwidth to be allocated, branch 1207 is 
taken. In decision block 1217, it is determined whether there 
are any bit streams 109(0 f° r which a minimal bandwidth 
must still be allocated. If there are none, branch 1219 is 
taken and the remaining bandwidth is returned at 1235. If 
there are still bit streams 109(0, the program takes branch 
1221 and enters the panic process 1223, which deals with the 
problem as required by the priorities of bit streams 109(0 . 
. . n) and then returns the remaining bandwidth at 1235. 
Similarly, branch 1229, taken when all priority levels have 
40 been processed, returns the remaining bandwidth at 1235. 
Continuing with panic process 1223, if a bit stream 109(0 
cannot receive the minimum rate it requires, one of two 
things may occur, depending on the bit stream: 
SMB 507(0 mav overflow, causing loss of data. 
45 bit buffer 119(0 m decoder 115(0 ma Y underflow, causing 
interruption of the display of pictures. 
In the first case, either the input to SMB 507(0 must be 
decreased or the output from bit SMB 507(i) must be 
increased. Generally, the second solution can be employed 
50 in the short term and the first in the longer term. Beginning 
with the second solution, the extra bandwidth must be taken 
from priority 2 and 3 bit streams, beginning with bit streams 
109(0 w * tn priority 3. These bit streams have no time 
constraints and can be denied any bandwidth at all for as 
long as is necessary. Bandwidth can also be taken from 
priority 2 bit streams 109(0 that have space in their SMBs 
507(0 DV having them output a repeat of a picture until the 
panic condition is over or until their SMB 507(0 threatens 
to overflow. Of course, what the repeat produces at the 
receiver is a still picture. Because the repeat picture is totally 
redundant with regard to the picture it is repeating, it always 
has fewer bits than that picture. 

Given that the reason for the substitution is to free up 
bandwidth, it is desirable to make the repeat picture as small 
as possible. That is achieved by sending a repeat of a coded 
65 picture that is not used to predict other pictures. B pictures 
fulfill this criterion, as do P pictures that immediately 
precede an I picture in sequences that do not contain B 
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pictures. The substitution technique requires that transmis- 
sion controller 413 for a PL 2 bit stream respond to an 
indication of a panic from central bitrate controller 1007 by 
reading header information to determine the type and size of 
the picture being output and when it finds the proper kind of 
picture, following it with repeat pictures until the panic is 
over. 

Where the problem is underflow of bit buffer 119, if the 
bit stream is a priority 1 bit stream, extra bandwidth must 
again be found and the techniques described above must be 
applied. If bit stream 109(0 is a priority 2 bit stream, the 
techniques described for priority 1 bit streams may be 
employed, or if that is not possible, the bandwidth required 
for the bit stream may be reduced by outputting a minimal- 
sized repeat picture as described above until the panic 
condition is over or until overflow of SMB 507(i) threatens. 

Where the problem is the threatened overflow of one or 
more SMB buffers 507, it may also be addressed by decreas- 
ing the bit rate at which the encoders 107 produce data. If the 
encoders 107 are co -located with statistical multiplexer 401, 
feedback from multiplexer 401 to the encoders may be used 
to do this. With this kind of feedback, there is no require- 
ment that multiplexer 401 understand the inner workings of 
encoders 107. All that the signal to a given encoder 107(f) 
need indicate is that the encoder must reduce its ouput rate 
by some amount. Which encoders receive the signal can be 
determined in many fashions by multiplexer 401. One 
approach is to reduce the bit rate (and therefore the image 
quality) in channels on the basis of their priority levels; 
another is to reduce the bit rate in all channels equally. 
Typically, taking bandwidth from other bit streams would be 
a short-term solution that would be employed until the 
encoding rate could be changed. In the preferred hardware 
embodiment, short-term panic management is done in cen- 
tral bitrate controller 1007, while long-term panic manage- 
ment is done in control processor 907. 
Algorithm for allocating residual bits 

When each of the bit streams 109(() has received its 
minimum bitrate and there is still bandwidth remaining in 
medium 207, this residual bandwidth B. is allocated among 
the bit streams in the preferred embodiment by allocating 
each bit stream 109(C) an additional bit rate AR(i) which is 
proportional to the difference between the maximum and 
minimum bit rates computed by TRC 413(f) for the bit 
stream. AR(i) is calculated in the preferred embodiment as 
follows: 

g w (0-/U,<0 Be 
{t) ~ £(ff«r(0-*«f.(0)Tc 

In a preferred embodiment, all of the bit rates involved in 
the above computation are rounded to an integer number of 
packets per second. 
Conclusion 

The foregoing Detailed Desrpfion has disclosed to those 
skilled in the arts to which the invention pertains how to 
make and use apparatus for outputting a variable-rate bit 
stream which uses a model of a receiver of the bit stream and 
information from the bit stream to determine a rate at which 
the variable-rate bit stream must be output to avoid overflow 
or underflow in the receiver. The Detailed Description has 
further disclosed a multiplexer which employs the apparatus' 
to multiplex a set of variable-rate bit streams onto a medium, 
has disclosed how such apparatus and multiplexers may be 
used with bit streams of video images that are encoded 
according to the MPEG-2 standard, and has given algo- 
rithms for the use of models of MPEG-2 receivers to 
compute rate requirements. 

The Detailed Description has disclosed the best mode 
presently known to the inventors of implementing their 
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apparatus and multiplexers which employ the apparatus; it 
will, however, be immediately apparent to those skilled in 
the arts to which the invention pertains that the invention 
may be employed with variable-rate bit streams other than 
5 those which are encoded according to the MPEG-2 standard 
and that many other implementations can be made which 
will incorporate the principles of the invention but employ 
different specific implementation techniques. 

For these reasons, the Detailed Description is to be 
J0 regarded as being in all respects exemplary and not 
restrictive, and the breadth of the invention disclosed herein 
is to be determined not from the Detailed Description, but 
rather from the claims as interpreted with the full breadth 
permitted by the patent laws. 
What is claimed is: 
15 1. A method used in a multiplexer that multiplexes a 
plurality of variable-rate bit streams onto a medium for 
allocating the bandwidth of the medium among the bit 
streams for a period of time, each bit stream having upper 
and lower bounds for the bit rate of the bit stream during the 
20 period of time and the method comprising the steps of: 
for each bit stream, allocating bandwidth from the 
medium as required for the lower bound of the bit rate 
of the bit stream during the period of time; 
determining how much bandwidth remains in the medium 
25 for the period of time; and 

determining the difference between the upper and lower 
bounds for the bit stream and for each bit stream, 
allocating a portion of the remaining bandwidth to 
the bit stream in proportion to the difference, such 
30 that for each bit stream, the bit rate of the bit stream 

during the period of time is no more than the upper 
bound. 

2. The method set forth in claim 1 wherein the step of 
allocating a portion of the remaining bandwidth to the bit 

35 stream includes the step of allocating bit stream i's portion 
AR(i) according to the formula 

w ~ £(K«„(0-*™<i))rc 

40 

where B c is the residual bandwidth for the period of time and 
T c is the period of time. 

3. A multiplexer for multiplexing a plurality of variable - 
45 rate bit streams onto a medium, the multiplexer being 

characterized in that: 

the multiplexer allocates bandwidth belonging to the 
medium to the variable-rate bit streams according to 
any of the methods set forth in claims 1 or 2. 
50 4. Apparatus for providing a plurality of variable-rate bit 
streams to a medium, the apparatus comprising 

a plurality of encoders for receiving constant-rate bit 
streams and producing variable-rate bit streams 
therefrom, 

55 a multiplexer for receiving the variable-rate bit streams 
from the encoders, and 
a transmitter for receiving the variable-rate bit streams 
and outputting the variable-rate bit streams to the 
6(J medium, the apparatus being characterized in that 
the multiplexer is a multiplexer according to claim 1. 
5. The method set forth in any of claims 1 or 2 wherein: 
at least one of the bit streams in the plurality thereof is a 
bit stream produced by encoding a sequence of video 
65 images according to the MPEG-2 standard. 

* * * * ♦ 
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